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Abstract — Computer-Based English Summative Test 
(CBEST) has gained popularity in many educational 
institutions recently. In this testing mode, students do the 
exam with the help of computer in language laboratory. 
The result of the test is usually gained immediately after 
the test is done. Though CBEST has been popular, the 
implementation of CBEST in Indonesian educational 
context has not been widely published. This paper 
specifically presents teachers’ beliefs and experiences in 
doing CBEST testing mode. The data were collected by 
distributing questionnaire to 9 (nine) English teachers 
from 3 (three) different vocational schools. The result of 
the study showed that teachers believe that the 
implementation of CBEST give them benefits in several 
aspects, such as aspect of economy, aspect of system 
implementation and aspect of test administration and 
design.. 
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I. INTRODUCTION 

In this 21 st century, computer has been one of tools which 
is usually used in education. It is not only used for the 
purposes of learning activities, but also used in testing 
purposes. In the context of English language teaching 
(ELT), the use of computer in language testing is popular 
with the term: Computer Assisted Language Testing 
(CALT). Specifically, when CALT is done during 
summative tests, this terminology is also known as 
Computer-Based Summative Test (CBST). In other 
words, CALT is a type of testing procedure in which 
language performance is assessed using computer. Three 
processed were involved during the implementation of 
CALT: generating the test, interaction with the 
candidate, and evaluation of responses (Noijon, 2012). 
In the first process, computer selects a number of items 
from an item bank, randomly or following some selection 
procedure until a number of items have been collected 
and for use in the constmction of a test. In the second 
procedure, test candidate do the test by using certain 


instructions in the computer, hi the third procedure, all 
candidates’ data which have been taken during the 
previous process are called up for a final evaluation of the 
complete response. 

In Indonesia, especially in vocational school 
context, CBEST is considered as a new testing mode. 
Based on the data from Indonesian Ministry of Education 
and Culture (Kemendikbud), in 2017, there were 9.829 
schools which has already implemented CBEST, while in 
2016, there were 2100 schools and in 2015, there were 
379 schools. Since the number of the school with CBEST 
testing mode is increasing each year, it is then important 
to indentify how teachers believe the new testing mode, 
and how do they prepare their students for facing the 
CBEST. Therefore, this study focuses on the 
implementation of CBEST in vocational school context 
by investigating teachers’ beliefs and experiences in the 
testing mode. 

II. METHODOLOGY 

Research Site 

This research was conducted in three different vocational 
schools in Bandung. They are: SMK Assalam, SMK 
Negeri 9 Bandung and SMK Sandy Putra Telkom SMK 
Assalam is located on Jalan Situ Tarate, Cibaduyut, 
Bandung. SMK Negeri 9 Bandung is situated on Jalan 
Soekamo Hatta, Bandung. While SMK Sandy Putra 
Telkom is located on Jalan Bojong Soang Bandung. 

From each school, three English teachers were 
involved. Since 2016 the schools have become one of 
adopter schools for Cambridge. In the process of teaching 
and learning English, the schools combine the 2013 
curriculum in combination with several Cambridge 
books, like Interchange, etc. The schools have already 
implemented Computer Based English Summative Test 
(CBEST). This summative test for English is delivered 
via computers conducted in language laboratory. 
Participant 

There are 9 participant teachers involved in this 
study. Three participants are English teachers from SMK 
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Assalam Bandung. Three other participants come from 
SMK Negeri 9 Badung. And the other three participants 
are from SMK Sandy Putra Telkom The more detailed 
data about the participants are displayed in the table 1 
below: 


Table. 1: Profiles of Participants 


N 

0 

Participant 

Age 

Qualificatio 

n 

Experience 

i 

Teacher 1 

(F) 

36 

year 

s old 

SI English 
Education 
Department 

12 years of 
teaching 
English 

2 

Teacher 2 

(F) 

41 

year 

s old 

SI English 
Education 
Department 

17 years of 
teaching 
English 

3 

Teacher 3 

(F) 

34 

year 

s old 

SI English 
Education 
Department 

10 years of 
teaching 
English 

4 

Teacher 4 

(F) 

46 

year 

s old 

SI English 
Education 

Department 

22 years of 
teaching 
English 

5 

Teacher 5 

(F) 

49 

year 

s old 

SI English 
Education 

Department 

24 years of 
teaching 
English 

6 

Teacher 6 

(F) 

44 

year 

s old 

SI English 
Education 
Department 

20 years of 
teaching 
English 

7 

Teacher 7 

(M) 

33 

year 

s old 

SI German 

Education 

Department 

9 years of 
teaching 
English 

8 

Teacher 8 

(M) 

39 

year 

s old 

SI French 

Education 

Department 

15 years of 
teaching 
English 

9 

Teacher 9 

(F) 

27 

year 

s old 

SI English 
Education 
Department 

3 years of 
teaching 
English 


Instrument 

In this research project, the researcher used questionnaire 
to gain the data. Questionnaires were distributed to 9 
participant teachers. “Questionnaires are any written 
instruments that present respondents with a series of 
questions or statements to which they are to react either 
by writing out their answers or selecting from among 
existing answers” (Brown, 2001). The researcher 
considers this data collection technique is the most 
suitable technique to elicit teachers’ beliefs and 
experiences in preparing and taking Computer-Based 
Summative English Tests (CBSET). This is because 
questionnaire can measure attitudinal information by 
asking attitudinal questions. Domyei (2009) elaborates 
that attitudinal questions are used to find out what people 
think. This is a broad category that concern attitudes, 
opinions, beliefs, interests and values. 


The use of questionnaire in this research project has 
several advantages. The first issue is cost-effectiveness. 
Questionnaire can save researcher time, effort and 
financial resources. By administering questionnaire to the 
a class of students which consists of this research, a huge 
amount of information about teachers’ beliefs and 
experiences about CBEST in vocational school levels can 
be collected in a relatively short time. The second issue is 
versatility. Domyei (2009) explains that questionnaire can 
be used successfully with a variety of people in a variety 
of situations targeting a variety of topics. This is also in 
accordance to what Bryman (2008) says that a well- 
constructed questionnaire can reduce the bias of 
interviewer effects and thus increase the consistency and 
reliability of the results. Due to these merits, it is no 
wonder that many research projects in social sciences 
employ questionnaire as one of the data collection 
techniques. The questionnaire consisted of 8 open-ended 
questions about teachers’ beliefs and experiences of 
CBEST. 

Procedure 

The first step in the research procedure was 
collecting the data. The data were collected by 
distributing questionnaire to 9 participants from three 
different vocational schools. The next process after 
collecting the data is to analyze it. Bums (2000) says that 
the purpose of analyzing the data is to find meaning in 
the data and this is done by systematically arranging 
and presenting the information. It has to be organized so 
that comparisons, contrasts, and insights can be made and 
demonstrated. Specifically, thematic analysis was used in 
analyzing the data. Boyatzis (1998) elaborates thematic 
analysis as a strategy in qualitative research to analyze 
information in a systematic way in order to make the data 
understandable. It organizes and describes the data in 
detail according to emergent themes, hi doing thematic 
analysis, the researcher used the phases by Braun and 
Clarke (2006) as the guidance. Table below displays the 
phases of thematic analysis process: 


Table.2: Phases of thematic analysis (Braun and Clark) 


NO 

PHASES 

DESCRIPTION 

1 

Familiarizing with 
the data 

Transcribing data, 

reading and re-reading 
the data, noting down 
initial ideas 

2 

Generating initial 
codes 

Coding interesting 

features of the data in a 
systematic fashion across 
the entire data set, 
collating data relevant to 
each code 

3 

Searching for 
themes 

Collating codes into 

potential themes, 

gathering all data 
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relevant to each potential 
theme 

4 

Reviewing themes 

Checking the themes 
work in relation to the 

coded extracts (level 1) 
and the entire data set 
(level 2), generating a 
thematic ‘map’ of the 
analysis 

5 

Defining and 

naming themes 

Ongoing analysis to 
refine the specifics of 
each theme, and overall 
story the analysis tells, 
generating clear 

definitions and names 

for each theme 

6 

Producing the 
report 

The final opportunity for 
analysis. Selection of 
vivid, compelling extract 
examples, final analysis 
of selected extracts, 
relating back of the 
analysis to the research 
questions and literature, 
producing a scholarly 
report of the analysis. 


The first step in the process of data analysis was 
organizing the data. This step involved transcribing the 
data gained from questionnaire. The data then were 
analyzed and interpreted to identify the links between the 
data in every questionnaire. After transcribing and 
translating the data, the next procedure was labeling 
the data based on the data sources. The next analytic 
procedure was repeatedly reading of the text of the 
transcription of the questionnaire. The next step was 
coding. Creswell (2009) states that coding process is to 
make sense out of data, divide it into text or image 
segment, label the segments with codes, examine codes 
for overlap and redundancy and collapse these codes into 
broad themes, hi this study, coding was intended to 
identify certain ideas in the data that represented the same 
meanings. Finally, the data were categorized into the 
aspects related to teachers’ beliefs and experiences in the 
implementation of Computer Based English Summative 
Tests (CBEST). 

III. RESULT AND DISCUSSION 

The elaboration about the result of this study is based on 
each question posed about teachers’ beliefs and 
experiences in CBEST from three different aspects: 
economy, system implementation, administration and 
design. 

Economy Aspect Of CBT 
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Questions 

Answers 

Yes 

No 

NA 

CBT testing mode 

9 

0 

0 

saves time 

(100%) 

(0%) 

(0%) 

CBT promote eco- 

9 

0 

0 

friendly environment 

(100%) 

(0%) 

(0%) 

CBT cost is cheaper 

6 

3 

0 

than PBT 

(67%) 

(33%) 

(0%) 


(Table 3: Teachers’Beliefs and Experiences of CBT from 


Economy Aspect) 

As we can seen from table 3 above, the majority of 
participants (100%) state that CBT testing mode can save 
times. They said that CBT testing mode can be done is 
shorter duration than PBT. In PBT, one subject test 
usually take 90-120 minutes. However, in CBT test 
duration can be made shorter to approximately 60 
minutes, because students just need to click the answers 
without blackening the answer sheets like in PBT. The 
second question is about whether CBT can promote eco- 
friendly environment. All participants agree that CBT can 
promote greener environment since it is paperless. No test 
booklet needed, no answer sheets needed. All scoring 
mechanism is done by computers. And the last question is 
about whether CBT is cheaper than PBT. The participants 
showed different beliefs. Three participants (33%) said 
that CBT is more expensive, since schools have to 
provide a number of computer units to facilitate students 
in doing CBT. Sometimes the number of the computer is 
not balanced with the number of students. And 6 other 
participants said that CBT is cheaper because schools can 
reduce the cost for printing test booklets, printing answer 
sheets, paying teachers for scoring students’works and so 
on. 


System Implementation Aspect Of CBT 

Questions 

Answers 

Yes 

No 

NA 

CBT has efficient 

8 

1 

0 

administration 

(89%) 

(11%) 

(0%) 

CBT save exam data 

9 

0 

0 

accurately 

(100%) 

(0%) 

(0%) 

In CBT, data can be 

9 

0 

0 

retrieved easily 

(100%) 

(0%) 

(0%) 


Viewed from aspect of system implementation, 
participants showed different beliefs. The majority of 
participants (89%) said that CBT has efficient 
administration since schools do not need to print the test 
booklets, store it in one room before distributing them to 
every classroom. However, there is one participant (11%) 
believed that CBT is not efficient since the school 
becomes dependent on IT experts. Once there is no IT 
experts, CBT will not run effectively. The second 
question is about whether CBT save accurate data 
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collection. All participants (100%) believed that in CBT, 
exam data will be more likely accurately collected and 
easier to store. Also, results and other data can be stored 
in much less space and it is easier to retrieve. Moreover, 
responses generally are accurately captures and scored, hi 
PBT however, students often make some marks on the 
answer sheets which can result in inaccurate scoring. . 


Test Administration & Design Aspect Of CBT 

Questions 

Answers 

Yes 

No 

NA 

CBT gives immediate 

9 

0 

0 

feedback 

(100%) 

(0%) 

(0%) 

CBT is preferred by 

5 

4 

0 

students 

(56%) 

(44%) 

(0%) 


Viewed from the aspect of test administration 
and design, all participants (100%) believed that CBT is 
effective because it can give immediate feedback. In PBT, 
the result of summative tests usually takes several days to 
finish. Teachers as testraters often need time to assess the 
test and to make decision about the tests. However, in 
CBT immediate viewing of scores on screen is provided 
in CBT to give test takers the instant feedback. This is in 
line with Mojarrad et al (2014) that immediate feedback, 
accurate test result reports and the possibility of printing 
the basic testing statistics are other advantages of using 
computer in assessment field that enable test takers take 
the test at any. The second question is about whether CBT 
is preferred by students. 5 participants (56%) said that 
students like CBT than PBT because many students are 
already familiar with the use of computer. Some students 
have used computers to play games and some of them 
might receive the instmction through computers. Some 
students might prefer CBT since they can customize the 
assessment based on their personal preference, like colors 
on the screen, font types, font sizes, and so on. Due to the 
possibility of customizing the assessment based on 
personal preferences, some people prefer to take CBT 
version of the test. However, 4 participants (44%) said 
that CBT is not always preferred by students. Some 
students are still confused how to use computers and they 
feel anxious when dealing with computers, so some 
students will perform better in PBT mode. This is in line 
with what Cater et al. (2010; Russell et al. (2010) that 
although some students may prefer CBT, others may 
prefer paper and pencil-based test. Some test takers prefer 
paper-based testing process because they are accustomed 
to taking notes and circling questions and answers for 
later review. Therefore, training for students before the 
CBT is needed to be conducted so students will be 
familiar with the computer and will feel comfortable in 
doing the test. 
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CONCLUSION 

The aim of this study is to investigate teacher’s beliefs 
about CBT in English testing. Their beliefs are based on 
their experiences in preparing students to face CBT. 
Teachers’ beliefs are based on the three aspects of CBT: 
economy, system implementation and test administration 
& design of CBT. The result of the study showed that 
teachers’ have different beliefs about the implementation 
of CBT. However, CBT can be one of the solutions in 
preparing students in facing the 21 st century needs by 
having technology literacy. 
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